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Abstract 



Much has been written about interviewing supervision but reports of 
empirical studies are rare. Rather than prescribing supervision behavior based 
on a theory, this report describes effective supervision based on behavioral 
reports by medical students. This study empirically identified the supervision 
behaviors that students considered effective and organized these behaviors into 
categories representing the important requirements for effective supervision. 
Also, the study compared the requirements for effective supervision by family 
physicians and behavioral scientists. Eighty-four sophomore medical students 
completed a "critical incident" report form at the end of an interviewing 
practicum session during which they were co-supervised, in groups of four, by a 
family physician and a behavioral scientist. Working independently, eleven 
judges sorted the reports of effective and ineffective supervision behavior by 
physicians and behavioral scientists "into categories. The eleven sets of 
categories were statistically pooled into joint proportion matrices. Principal 
component analyses of the matrices extracted the source variables that underlie 
the observed behaviors and constitute the important requirements of supervision. 
The requirements of effective supervision by physicians were to model the 
physician's role, teach clinical problem-solving, provide balanced feedback, 
suggest how to improve interviews, structure the task, prod critical thinking, 
aiijd confront defensiveness. The requirements for effective supervision by 
behavioral scientists were to teach interviewing and interpersonal skills, provide 
constructive criticism, discuss patient behavior, convey personal support, 
promote change, and facilitate group interaction. The findings can be used to 
orient new faculty to the supervision task and to structure faculty development 
sessions for veteran supervisors. 
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Requirements for Effective Interviewing Supervision 

Providing supervision to students practicing interviewing and interpersonal skills is 
a common role among family medicine faculty. The present study sought to identify 
supervisory behaviors considered effective by medical students and to organize these 
behaviors into descriptive categories rep. 3senting the important requirements for 
effective supervisioh. In addition, the study compared the supervisory behavior of 
family physicians to that of behavioral scientists in order to examine their respective 
roles in supervision of medical students practicing interviewing and interpersonal skills. 

Method 

The sample consisted of 84 second year medical student'; participating in a required 
360-hour course on physical diagnosis, medical interviewing, and ambulatory care. The 
course included a 36-hour interviewing and interpersonal skills practicum. Students met 
in groups of four, for 12 three-hour sessions during a nine month period, to review 
videotapes of their interviews with simulated patients. The students also provided 
written medical records for review on each patient. Each group of four students was 
co-supervised by a family physician and a behavioral scientist who remained the same 
for the entire nine months. 

The critical incidents technique was used to collect reports of supervisory behavior 
through a 4-page report form developed using Flanagan's (1954) guidelines. Each page 
requested a single incident: physician effective behavior, physician ineffective 
behavior, behavioral scientist effective behavior, and behavioral scientist ineffective 
behavior. 

The incident reports were collected during the fourth meeting of each practicum 
group. With 20 minutes remaining in the 3-hour session, a secretary entered the room 
and reminded the supervisors of the study. The supervisors and the secretary then left 
the room while the students recorded the incidents on the report form. Each student 
returned the form, without identifying information, to the secretary. 

The data analytic procedures were designed to empirically extract the important « 
requirements of effective supervision from the reports of critical supervisory behaviors. 
Each incident was typed on an index card. There were four sets of cards 
corresponding, respectively, to effective and ineffective behavior of physicians and 
behavioral scientists. Rather than convening a panel of judges to work together on the 
•sorting task, eleven judges worked independently. The judges were instructed to sort 
each of the four sets of incidents into mutually exclusive, jointly exhaustive categories 
on the basis of the supervisory behavior. The instructions carefully avoided suggesting 
a particular number of categories or what type of categories were expected. After 
categorizing the behaviors, each sorter re-examined the categories and refined them so 
that all items in each category were homogeneous. Finally, each sorter assigned a title 
to each category. 

The eleven sets of subjectively-derived manifest categories were statistically 
pooled to create a joint proportion matrix. The joint proportion matrix was analyzed 
with the principal component analysis subprogi^m PM of the Statistical Package for 
the Social Sciences (Nie, Hull, Kenkins, Steinbrenner & Brent, 1975). Considering the 
large size of the matrix (84 x 84), it was anticipated that using latent roots greater 
than one as the criterion for number of components would result in numerous small, 
trivial components. Accordingly, the scree test was iLsed to indicate the effective 
number of large, substantive components accounting for most of the variance while 
minimizing^ the number of components. To enhance interpretability, the components, 
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were rotated to simple structure using varimax criterion. The resulting component 
loadings were treated as latent category membership coefficients. The latent 
categories were named after examining the titles of the sorters' manifest categories 
for each behavior that had a category membership coefficient exceeding .30. The titles 
of these manifest categories were synthesized to name the latent category. 

The categories of this hypothetical latent set represented tiie source variables 
underlying the observed behaviors. Collectively, these categories constituted the 
important requirements for effective supervision of beginning practicum students by 
physicians. This data analytic procedure then was applied to the other three sets of 
reports: physician ineffective, behavioral scientist effective, and behavioral sciei.iist 
ineffective. 

Results 

Physician Effective Behaviors 

The 11 sorters used from five to 17 categories (x = 10.4; S.D. = 2.4) to group the 
84 critical behaviors reported by students as effective supervisory acts by their 
physician supervisors. Principal component analysis of the joint proportion matrix 
constructed from these 11 sets of manifest categories extracted 21 components with 
latent roots exceeding 1.0. They accounted for 82.6% of the variance. As anticipated, 
inspection of the 21 components revealed numerous small, trivial components. In fact, 
10 of the 21 components had only one or two behaviors loading .30 or higher. The 
scree test indicated seven substantive components. The seven components accounted 
for 56.6% of the total variance. Accordingly, the matrix was reanalyzed with PAl 
solving for seven components. The resulting seven latent categories were titled and 
interpreted. 

Physician Ineffective Behaviors 

Of the 84 report forms, 30 stated that the physician had not displayed any 
ineffective behavior during that practicum session. Sixteen of these forms did report an 
ineffective behavior by the behavioral scientist co-supervisor. Therefore, it was 
assumed that lack of a report did not reflect an uncritical observer. Only the 54 forms 
reporting an ineffective behavior were analyzed. The 11 sorters used from four to 14 
categories (x = 9.36; S.D. = 1.69) to group the 54 behaviors. Principal component 
analysis of the joint proportion matrix constructed from the 11 sets of manifest 
categories extracted 13 components with latent roots exceeding 1,0. The 13 
components accounted for 77% of the variance. Based on the scree test, a forced seven 
component solution was computed with PAl. The seven components accounted for 
61.3% of the total variance. 
Behavioral Scientist Effective 

The 11 sorters used from six to 13 categories (x = 9.6; S.D. = 1.25) to group the 84 
critical behaviors reported by students as effective supervisory behavior by behavioral 
scientists. Principal component analysis of the joint proportion matrix constructed from 
the 11 sets of manifest categories extracted 26 components with latent roots exceeding 
1.0. The 26 components accounted for 99.1% of the variance. Based on the soree test, 
a seven component solution was computed. The seven components accounted for 59.9% 
of^the total variance. 
Behavioral Scientist Ineffective Behaviors 

Of the 84 student responses, 31 stated that the behavioral scientist had not 
displayed an ineffective behavior during that practicum session. Seventeen of these 
reports noted an ineffective behavior in the physician supervisor. Therefore, it was 
assumed that lack of a report di'J not reflect an uncritical observer. Only the 53 
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reported ineffective behaviors were analyzed. The 11 sorters used from four to 14 
categories (x = 10.6; S.D. = 2.3) to group the 53 behaviors. Principal component 
analysis of the joint properties matrix constructed from the 11 sets of manifest 
categories extracted 15 components with latent roots exceeding 1.0. The 15 
components accounted for 77.4% of the variance. Based on the scree test, a 
six component solution was computed. The six components accounted for 53,2% of the 
total variance. 

Table 1 shows the categories within each of the four groups, along with the 
percent of variance accounted for by each category and the number of behaviors in 
each category. A detailed description of each category appears in the Appendix. 

Discussion 

What did medical students consider requirements for effective practicum 
supervision? For physician supervisors, the primary requirements were to model the 
physician's role, teach clinical problem-solving, provide balanced feedback, and suggest 
how to improve interviews. Additional categories of effective behavior were 
structuring the task, prodding critical thinking, and confronting defensiveness. Modeling 
was important to students in beginning practicum because they were struggling to 
adopt the physician's role. The students perceived physicians talking about pertinent 
experiences from their practices as an effective means of expanding students' 
interviewing repertoire. Yet, "storytelling" or discussing irrelevant expe"iences was 
clearly judged to be an ineffective behavior. The students also valued didactic teaching 
of clinical problem-solving strategies. Students needed more than evaluation of their 
performance. They wanted to learn cognitive schemes as well as specific behaviors that 
increased their competence. Moreover, they wanted the physician to structure their 
learning by identifying essential objectives and skills. When viewing their taped 
interview, students appreciated balanced feedback that attended to both their 
strengths and weaknesses from an informed perspective. They wanted supervisors to 
pay close attention to their tapes and carefully evaluate their write-ups. Even so, 
students did not wish to be overly dependent on supervision. Accordingly, they prized 
behaviors that promoted critical thinking or confronted defensiveness. 

Students reported different requirements for effective supervision by behavioral 
scientists. The primary requirements were to teach interpersonal and interviewing 
skills, provide constructive criticism, and offer positive reinforcement. Students also 
valued behaviors that conveyed personal support or facilitated group discussion. With 
regard to teaching, students considered instruction in understanding patient behavior as 
a requirement of effective supervision. Students valued learning how to understand a 
patient's character. On the other hand, suE>ervisor characterizations of patients that 
seemed stereotyped or limiting were judged as quite ineffective behavior. In addition to 
instruction in the "content" of patient behavior, students liked behaviors that taught 
interviewing "process" skills. They sought instruction in interviewing techniques and 
wanted encouraging feedback on their taped interviews. In their view, effective 
feedback was accurate, specific, and reinforcing. Besides feedback concerning 
professional skills, students valued behaviors that were supportive of them as 
individuals. Students appreciated acts that increased their self-confidence or reassured 
them. This support facilitated student comfort in adopting and experimenting with the 
student-physician role. Students also prized efforts to facilitate a "group experience" 
among the four students in a practicum. 

Contrasting the student-perceived critical requirements of physician versus 
behavioral scientist supervisors revealed three major differences. First, both 
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supervisors provided instruction but generally on different topics. Physicians taught 
biomedical information and clinical problem-solving. Behavioral scientists taught 
interviewing and psychosocial assessment of patients. A further difference was that 
physicians appeared to structure the discussions while behavioral scientists facilitated 
active group participation. As topics were discussed, physicians seemed to stress 
biomedical competence while behavioral scientists stressed psychosocial sensitivity. 

Second, both supervisors provided students with feedback on their taped interviews. 
When they differed, it appeared that physicians tended to identify deficiencies, demand 
excellence, and confront excuses. Behavioral scientists seemed more likely to offer 
reinforcement and encouragement. Admittedly, the students wanted feedback on their 
successes and failures from both supervisors. When they did not encounter this balance, 
students tended to perceive physician feedback as harsh and behavioral scientist 
feedback as overly kind. 

The third difference between supervision requirements of physicians and behavioral 
scientists involved helping supervisees make the transition from medical student 
learning facts to student-physician starting to apply these facts in clinical work with 
patients. Physicians aided students by serving as role models and by talking about their 
experiences. Obviously, behavioral scientists could not serve as physician role models. 
Instead they offered support to students as individuals changing roles. In addition, 
behavioral scientists facilitated group discussion of the role transition and related 
issues while mobilizing social support from the group. 

Based on the above three differences, it was concluded that the requirements for 
effective supervision of the two types of supervisors revealed both style and content 
differences. On 'jalance, when the two differed, physicians leaned toward a more 
instrumental style of supervision in contrast to the behavioral scientists' more 
affective style. Physicians more often highlighted professional development aspects of 
the task. Behavioral scientists tended to underscore personal growth issues inherent in 
the task. With regard to substantive content, differences may be ascribed to 
physicians' focus on the biology of disease jn contrast to behavioral scientists' 
attention to the psychology of illness. Both supervisors, however, emphasized that 
primary-care medicine requires both curing and caring. 

In considering the results of this study, one should remember that the requirements 
for effective supervision evolved from student perceptions. While student perceptions 
of supervision are important, they should not be interpreted without considering the 
students' level of professional development. As students in a beginning practicum, they 
may have tended to report supervision behaviors that responded to their needs for 
structure and reassurance. While these behaviors vould still be considered examples of 
effective supervision, they might relate more to student satisfaction (e.g., comfort in 
the situation, liking for the supervisor) than to student performance. In fact, 
examination of the requirements listed in Table 1 suggested that most of the 
student-perceived requirements seem to coincide with evaluation and support. 
Moreover, the specific behaviors reported were quite concrete, and focused more on 
substantive content and technique than on dynamic process <ind self-discovery. 
Supervisors viewing these same practicum sessions may have reported more behaviors 
that dealt with process issues and student performance than the sample of students 
reported. Further study is needed to investigate the link between student-perceived 
requirments for effective supervision and student performance, as well as the 
relationship between student- and supervisor-perceived requirements for effective 
supervision. 



ERIC 



7 



n 



References 

Flanagan; J. C. The critical incident technique. Psychological Bulletin . 1954, 51, 
327-358. • 

Nie, N., Hull, C, Kenkins, J., Steinbrenner, K. & Brent, C. Statistical Package 
for the Social Sciences. New York:MeGraw-Hill, 3975. 



c 



ERIC 



8 



TABLE 1 

Latent Category Titles, Variance Accounted Fop, and 

Number of ttems 



Category Title 



Physician Effective 

1. Model 

2. Teach Problem-solving 

3. Provide Balanced Feeduack 

4. Suggest Alternatives 
' 5. Identify Essentials 

6. Prod Critical Thinking 

7. Confront 

Physician Ineffective 

1. Wordy 

2. Inattentive 

3. Uninformed 

4. Harsh 

5. Uncritical 

6. Ignore Reports 

7. , Misplace Emphasis 

Behavioral Scientist Effective 



Percent of 
Variance 



19.7 
14.2 
8.8 
4.3 
3.6 
3.2 
2.8 



16.3 
12.8 
10.2 
8.7 
5.0 
4.3 
4.0 



Number of 
Behaviors 



14 
23 
17 



6 
7 
6 



10 
13 
7 
5 
6 
V5 
6 



A- 



1. Teach Interpersonal Skills 23^.3 

2. Provide Encouraging Feedback 11.6 

3. Teach Interviewing Skills 6.9 

4. Promote Change 5.8 

5. Openness 5.0 

6. Encourage and Support 3.7 

7. Facilitate Group 3.6 

Behavioral Scientist Ineffective 



19 
15 
15 
10 

5 

11 
9 



1. Inattentive 

2. Inappropriate Feedback 

3. Reserved 

4. Vague 

5. Preoccupied 

6. Stereotyped Tfiinkiiig 



16.3 
13.3 
7.5 
5.9 
5.4 
4.8 



11 

14 
7 

11 
7 
3 
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APPENDIX 

PHYSICIAN EFFECTIVE BEHAVIORS 

1. Model. Inspection of the sorter's manifest categories in which these behaviors 
were placed revealed titles such , as "modeling", "role model", "self-disclosure", 
"disclosure", "sharing", and "sharing experiences". Thus, the important requirement was 
defined as "serve, as a role model by self-disclosing and sharing experiences from 
clinical practice". The reported behaviors constituting this category involved the 
supervisors taking about their own patients who were similar to the videotaped patient 
or using exampfe^ from fs^actice to illustrate or clarify points. The students considered 
this effective behavior because it offered a realistic perspective on jthe relevance of 
interviewing skills in medical practice. Moreover, students thought the behaviors 
modeled psychosocial Sx'^nsitivity. 

2. Teach problem- solving . The underlying requirement organizing this category of 
supervisory behavior was "teach case problem-solving skills". The reported behaviors 
consisted of teaching symptom pursuit, differential diagnosis, and management planning. 
The students viewed these as effective behaviors .because the acts reinforced and 
illustrated material learned in lectures. 

3. Provide balanced feedback . The latent requirement for the third category was 
"give positive feedback together with the negative feedback". The behaviors involved 
adding encouraging statements to critical feedback, noting positive aspects, of an 
interview, commenting on improvement from previous performance, and recognizing 
what had been learned as well as what still needed to be learned. These behaviors 
seemed to be valued by students because they reduced tension and built confidence. 

4. Suggest alternatives . The fourth requirement was "offer specific words, actions, 
or questions to be tried instead of or in addition to what the student did on the tape". 
The behaviors constituting this category were hints, suggestions, tips, or advice on how 
to improve. These behavior? were judged effective by students because they 
encouraged students to try different techniques, l^oreover, the supervisors offered 
these alternatives as suggestions rather than commands, thereby conveying a respect 
for the student's interviewing style. 

5. Identify essentials . The source requirement organizing this category was 
"emphasize what is important". Reported behaviors included stressing what is really 
important to accomplish and pointing out the most pertinent information gathered 
during a particular interview. Students liked these behaviors because the act focused 
student attention or reduced anxiety by structuring the tasks. 

6. Prod critical thinking . The sixth requirement of effective supervision was 
"demand clinical problem-solving and challenge thinking". These behaviors largely 
focused on reactions to student case presentations. For example, questioning the 
student, insisting on organized thoughts, asking students to critique their own 
presentation, and teaching clinical thinking techniques. Students appreciated these 
behaviors because the acts developed their problem-solving competence and reflected 
expectations for excellence. 

7. Confront . The final requirement was "ask students lo be honest and germine in 
relating to patients". The behaviors comprising this category consisted of pointing out 
incongruities in student actions and helping students take responsibility for their own 
feelings rather than blaming patients. The students considered these behaviors 
effective fc>ecause the behaviors made them more receptive to patients and more aware 
of how patients reacted to them. 
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PHYSICIAN INEFFECTIVE BEHAVIORS 

1. Wordy . The underlying requirement of this category was "do not belabor 
introductions or comments". The ineffective' J)ehaviors constituting this category 
consisted of talking too long on the same subject or on an insignificant topic, and 
repeating an idea or "mini-lecture" excessively. Students view such behavior as 
unnecessary elaborations. 

2. Inattentive . The organizing requirement weS "be on time and do not talk or 
write whik viewing videotapes". The ineffective behaviors composing this category 
were talking or filling in evaluation forms while a tape played and arriving late or 
leaving the room for a short period. Students considered these behaviors discourteous. 

3. Uninformed . The requirement was "know the students and their assignments for 
the session". The ineffective behaviors in this category centered on unfair criticism 
caused by lack of familiarity with the students or the curriculum. Examples include 
expecting students to know things they had not been taught and criticizing students for 
not doing what they had not been assigned. 

4. Harsh > . The requirement was "do not overlook successes and positive 
accomplishments". The ineffective behaviors in this category shared the theme of 
focusing exclusively on mistakes while not providing reassurance or encouragement. 

5. Uncritical . This category was the opposite of the prior one. The requirement was 
• "do not overlook errors or shortcomings". The ineffective behaviors constituting this 

category were denying or ignoring student incompetence and being overly diplomatic in 
providing feedback on errors. 

6. Ignore reports . The requirement for this category was "do not negle i written 
reports". Students judged failing to return write-ups, or returning them without verbal 
or written feedback, to be ineffective supervisory behavior. 

7. Misplace emphasis . The requirement reflected by this category was "do not 
concentrate on minor details". Students thought that emphasizing minor points, 
attending to picKy details, and wandering off the topic were ineffective behaviors 
because these actions misrepresent the essentials of successful interviewing. 

BEHAVIORAL SCIENTIST EFFECTIVE BEHAVIORS 

1. Teach interpersonal skills . The underlying requirement for this category was 
"teach how to use verbal and non-verbal cues to understand and classify patients". The 
behaviors constituting this category included instruction in how to observe and analyze 
patient behavior during an interview and advice on how to deal with certain types of 
patients. 

2. Provide encouraging feedback . The requirement underlying this category was 
"provide encouraging feedback on interviewing techniques". The specific behaviors 
consisted of recognizing improvements, emphasizing positive points, and offering tips on 
how to improve. Students appreciated these behaviors because they reflected a 
supportive attitude. 

3. Teach interviewing skills > The requirement for this category was "teach 
interviewing skills by explaining and modeling alternative approaches". Representative 
behaviors in this group included giving alternatives to inappropriate behavior, changing 
closed questions into open ones, and pointing out more effective actions than the one 
used . Students considered these behaviors effective because the instructions were 
specific and concrete. 

4> Promote change > The underlying requirement for tliis category was "encourage 
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skill development through constructive criticism and positive reinforcement". Behaviors 
that students thought facilitated change and experimentation consisted of offering 
corrective criticism while noting achievements and reinforcing effort. 

5.. Openness . This category was the only one in the study that was difficult to 
interpret. Only "five items had significant category membership coefficients. The two 
behaviors with the highest coefficients involved the supervisors self-disclosii^g problems 
they had in learning interviewing skills. The remaining three behaviors dealt with 
gsking open questions, avoiding double questions, and asking direct questions* 
Admittedly, construing an organizing requirement influencing all five behaviors was 
difficult. Nevertheless, a requirement organizing these behaviors will be offered: 
"model openness and teacn how to promote patient openness". 
, 6. Encourage and support . The requirement underlying this category was "encourage 
and support students as persons learning a profession". The behaviors forming this 
group were focusing on students^ feelings about their skills, maintaining eye contact, 
, using first names, promoting self -exploration, and jaccepting an occasional "bad day". 
Studenis considered these effective behaviors because the actions reflected empathy 
for students adopting the physician's role. 

7. Facilitate group . This category's requirement was "elicit affective and open 
communication among students". The effective behaviors consisted of noticing 
nonverbal communication, nurturing friendships among the students, promoting gtoup 
discussion, and encouraging self-disclosure and honesty. Students appreciated these acts 
because the acts tapped the potential of a "group experience". 

BEHAVIORAL SCIENTIST INEFFECTIVE BEHAVIORS 



1. Inattentive . The re<Dtiirement and specific behavior were the same as Category, 2 
y ~ for ineffective behlaviors by physician supervisors. 

2. Inappropriate feedback . The requirement organizing this category of ineffective 
behavior was "do not -misquote, prematurely criticize, or feel compelled to comment". 
Behaviors in this- group seemed to be interpreted by students as "overeagerness" to 
criticize. 

3. Reserved . The requirement influencing, this category was "do not be 
apprehensive in offering criticisms on unsatisfactory actions or portions of a tape". The 
behaviors included not talking much or not commenting on negative aspects of a 
student's performance. Students perceived reserved supervisors as being timid, 
apprehensive about b^ing critical, or intimidated by their co-supervisor. 

4> Vague. The requirement organizing this category was'^'do not be vague, abstract, 
or nonspeci^c". Behaviors constituting this category were commenting on a tape as a 
whole, criticizing an action without suggesting how to do it differently, and vague or 
general answers to specific questions. Students thought these behaviors reflected lack 
of active participation rather than lack of expertise. 

5. Preoccupied , The requirement underlying this category was "do not be 
preoccupied or easily distracted". The behaviors involved not concentrating on the task 
or being easily distracted. Students recognized that supervisors had other duties and 
responsibilities. Nevertheless, they wished supervisors would fully concentrate on the 
task at hand. 

6. Stereotyped thinking . The requirement generating this category was "do not 
overanalyze or stereotype patients". The behaviors included overinterpreting a patient's 
behavior, classifying people rather than their behavior, and characterizing patients in a 
restrictive manner. 
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